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ABSTRACT 

An  experiment  on  predicting  flood  flows  at  different  upstream  and  a  down  stream  section  of  a  river  network 
applying  partial  recurrent  neural  networks  (PRNN)  with  and  without  memory  structure  attached  to  the  input  layer  is 
presented.  Performance  of  PRNN  having  TDNN  memory,  Gamma  memory,  and  Laguarre  memory  attached  to  the  input 
layer  have  been  investigated  in  the  study.  The  models  are  applied  to  forecast  flood  flows  at  four  different  locations  in  Tar 
basin,  USA.  Results  obtained  indicates  that  though  there  may  be  difficulties  in  training  a  partially  recurrent  network  having 
no  memory  the  model  performs  better  in  forecasting  multiple  flows  over  a  basin. 

KEYWORDS:  PRNN,  Gamma  Memory,  River  Network 

INTRODUCTION 

Flood  flow  prediction  with  adequate  lead  time  is  essential  to  provide  time  for  taking  safety  measures  and 
evacuation  of  population  being  endangered  by  the  imminent  flood  disaster.  Artificial  Neural  Network  (ANN)  is  a  powerful 
computational  tool  having  capabilities  of  capturing  underlying  characteristics  of  a  physical  process  from  the  data  set. 
ANN  models  can  extract  relationships  between  the  input(s)  and  output(s)  efficiently  but,  the  model  can  not  explain 
underlying  physics  governing  the  problem.  Some  of  the  earliest  application  of  ANN  in  hydrology  has  been  reported  by 
Daniel  (1991).  Application  of  static  ANNs  in  modeling  hydrologic  time  series  can  be  found  in  the  works  of  Hsu  et 
al.(1993,1995),  Zhu  et  al.(1994),  Smith  and  Eli(1995),  Thirumalaiah  and  Deo  (1998),  Sajikumar  and 
Thandaveswara(1999),  Zeeland  et  al.(1999),Sudheer  et  al.(2002),  Chou  et  al.(2005),Deka  and  Chandramoulli(2005), 
Shrestha  et  al.(2005),  Alvisi  et  al.(2006),  Raiand  Mathur  (2007),  Kisi(2004,2007,2008).  The  ASCE  task  committee  (2000) 
on  application  of  ANNs  in  hydrology,  while  summarizing  the  application  of  ANN  in  different  areas  of  hydrology  opined 
that  ANN  may  be  suitable  for  the  problems  of  estimation  and  prediction  in  hydrology  because  of  its  strong  capacity  to 
identify  the  underlying  nonlinear  relationship  of  a  physical  process.  Application  of  ANN  in  areas  of  hydrology  shows  that 
mostly  static  ANN  models  have  been  employed  to  describe  hydrologic  problems.  Maier  and  Dandy  (2000)  reviewed  43 
papers  dealing  with  prediction  and  forecasting  of  water  resources  variables  and  reported  that  all  but  only  two  papers  used 
static  MLP  networks.  MLPs  and  other  static  ANN  models  apply  only  input  output  matching  to  describe  a  physical  process. 
Flood  flow  being  a  non-linear  time  varying  hydrologic  process  the  flow  parameters  changes  continuously  with  time  and  the 
variation  characteristics  for  a  reach  is  contained  in  the  data  set  collected  systematically  over  time.  Temporal  variations  in 
input  and  output  of  a  system  can  be  modeled  with  an  ANN  using  a  Time  Delay  Neural  Network  (TDNN).  A  TDNN  is 
essentially  equivalent  to  using  an  MLP  with  multiple  past  samples  as  input.  In  recent  past  Hsu  et  al(1997),  Anmala  et 
al.(2000),  Dibike  et  al.(1999),  Coulibaly  et  al.(2001),  Diamantopoulou  et  al. (2006), Wang  et  al.(2009)  applied  different 
types  of  recurrent  ANN  models  to  incorporate  temporal  dimensions  for  a  hydrologic  problem  and  the  results  are 
encouraging.  Most  of  the  ANN  based  flood  forecasting  models  available  in  the  literature  is  capable  of  providing  forecast  at 
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a  single  location  and  do  notpossess  forecast  updating  capability.  This  limits  applicability  of  the  models  in  real  time 
applications. 

During  flood,  flow  at  upstream  and  downstream  stations  change  continuously  with  time.  Flood  flow  in  a  river 
system  is  characterized  by  coherent  evolution  of  inflow(s),  outflow(s)  and  storages  in  the  river  reach  and  indicate  existence 
of  a  relationship  among  concurrent  flows  at  the  bounding  sections. 

A  partially  recurrent  neural  network  (PRNN)  possesses  recurrent  structure  connecting  the  first  hidden  layer  to 
itself  through  a  recurrent  connection  and  also  adds  a  feed  forward  connection,  through  a  bypass  synapse,  from  the  input 
axon  to  the  layer  after  the  first  hidden  layer.  In  the  case  of  PRNN,  the  recurrent  structure  acts  as  a  state  for  the  feed  forward 
structure  and  has  infinite  memory  depth.  On  the  other  hand  a  time  lagged  Recurrent  Neural  Network  (TLRN)  has  adaptable 
memory  depth  with  an  effective  upper  limit.  This  paper  investigates  applicability  of  PRNN  models  with  and  without 
memory  attached  to  the  input  layer  for  predicting  concurrent  flood  flows  at  multiple  locations  in  a  river  system. 
The  present  study  shows  a  PRNN  model  having  infinite  memory  depth  may  face  difficulties  during  training  but  can 
provide  better  results  in  forecasting  multiple  flows. 


Figure  1:  Map  of  the  Study  Area  in  Tar  River  Basin 


STUDY  AREA 

The  proposed  flood  forecasting  model  formulations  is  demonstrated  for  the  Tar -Pamlico  River  basin  in  North 
Carolina  (NC),  USA.  Figure  1  shows  the  map  of  the  study  area.  Major  tributaries  of  the  Tar  River  main  stem  include 
Cokey  Swamp,  and  Fishing,  Swift  and  Sandy  Creeks.  In  the  Upper  Tar  river  basin,  tributary  flow  from  Swift  Creek, 
Fishing  and  Little  fish  Creeks  contribute  to  the  flow  at  Tarboro,  NC.  This  study  uses  present  flow  information  available  at 
Tarboro  (Q4)  and  its  three  upstream  gauge  sites,  at  Rocky  Mount  (Qi),  Hilliardston  (Q2),  and  Enfield  (Q3)  to  predict  future 
flows  at  these  stations. 

PARTIAL  RECURRENT  NEURAL  NETWORK  (PRNN) 

A  Partially  recurrent  neural  network  feedbacks  the  first  hidden  layer  to  itself  through  recurrent  synapse  connection 
via  a  context  unit.  The  context  unit  represents  a  type  of  short  term  memory  operated  by  a  delay  operator.  In  addition  to 
feedback  the  network  also  adds  a  feed  forward  connection  from  input  axon  to  the  layer  after  the  first  hidden  layer  through  a 
synapse  bypassing  recurrency.  The  recurrent  connection  in  the  hidden  layer  allows  the  network  to  detect  and  generate  time 
varying  pattern  of  the  dynamic  system.  The  output  of  the  hidden  nodes  at  time  t  which  are  stored  in  the  context  units  are 
fed  back  as  additional  inputs  at  time  (t+1),  effectively  treating  the  recurrent  structure  act  as  a  state  for  the  feed  forward 
structure. 
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In  addition  to  recurrent  connection  in  the  hidden  layer  memory  can  be  attached  with  the  input  layer  of  a 
fully  /  partially  recurrent  network.  Three  types  of  memory  structure  can  be  found  in  the  literature  these  are:  TDNN, 
Gamma  and  Laguarre  memory.  TDNN  is  a  multi  channel  tapped  delay  line  with  ideal  delay  and  the  memory  is  generated 
by  a  time  delay  unit.  A  TDNN  has  fixed  memory  depth  with  highest  resolution,  jU  =1  while,  Gamma  and  Laguarre  has 
adaptable  memory  depth.  Adaptable  memory  depth  allows  deciding  best  duration  for  an  input's  past. 


y<t> 


Nemonj 


DeL^  line  of 
oinkr  K 

Figure  2:  Single  Neuron  with  Delay  Line  of  Order  K 

For  a  single  neuron,  i  at  the  input  layer  shown  in  Figure  2  for  any  given  input  Xi  (t)  ,  the  TDNN  delay  operator, 
T  yields  the  past  values  given  by,  Xik  (f)  =  X;  (t  —  k),k  =  0,1,2,3,  K . 

Gamma  memory  (GM)  cascades  self  recurrent  connections  with  local  feedback.  The  memory  generated  by  the 
time  delay  unit  has  a  adaptable  parameter  ^  which  can  control  the  memory  depth,  .  For  the  neuron,  *  shown  in  Figure  2 
when  the  input  layer  is  attached  with  gamma  memory  the  delay  is  a  dispersive  delay  line  of  order  K )  generated  by  delay 

unit  operator,  T .  There  exists  a  trade-off  between  memory  depth  and  resolution  parameter  ^  .  For  a  K   order  structure 

j  K 
the  memory  depth  a  =  —  . 

jU 


Multiple 

input 
variable  Delay 
line 


Generic  Output 
predictor  vector 


Figure  3:  Partial  Recurrent  Neural  Network  with  Focused  Memory 
Used  for  Prediction  of  Flow  at  Multiple  Sections 
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For  a  number  of  tap  used  in  the  memory  structure  resolution  parameter  jU  can  be  optimized  using  a  performance 
criterion.  Since  for  a  number  of  taps  in  the  memory  element  best  compromise  between  resolution  and  the  memory  depth 
can  be  achieved,  a  GM  can  decide  duration  that  best  represents  an  input's  past.  For  any  given  input  Xi  (t)  to  a  neuron  i  in 

the  input  layer,  the  value  for  the  first  tap  Xi0  (t)  =  Xt  (t)  and  delay  operator  yields  the  past  value  of  Xjk  (t) ,  given  by 

Xik  (?)  =  (1  -  ju)xik  (t  —  l)  +  lJXi  k_x  (t  -l),k  =  1,  K  and  are  stored  in  the  taps.  Figure  3  shows  the  PRNN  model  with 

the  memory  attached  to  the  input  layer  used  in  this  case. 

Laguarre  memory  (LM)  is  based  on  the  Laguarre  function  and  has  same  properties  as  that  of  gamma  memory. 
Similar  to  the  GM  the  resolution  parameter  is  used  to  control  memory  depth  for  a  number  of  taps  used  in  the  memory 
element. 


Rficunent 
connection 


ConteKt 
units 

Figure  4:  Partial  Recunent  Neural  Network  without  Attaching  Memory  to  the  Input  Layer  (PRNN) 

Four  different  PRNN  models  are  used  in  present  study  to  forecast  river  flows  at  multiple  sections  in  a  river 
network.  The  models  used  are  (i)  partially  recurrent  neural  network  with  focused  Gamma  memory  (PRNNG),  (ii)  partially 
recurrent  neural  network  with  focused  Laguarre  memory  (PRNNL),  (iii)  partially  recurrent  neural  network  with  focused 
TDNN  memory  (PRNNTD)  and  (iv)  partially  recurrent  neural  network  with  no  memory  (PRNN). 

NETWORK  DESIGN  FOR  PREDICTION 

The  computational  complexity  of  a  network  and  its  generalization  capabilities  directly  depends  on  network 
topology.  A  larger-than-necessary  networks  tend  to  over  fit  the  training  data  leading  to  poor  generalization  performance, 
while  on  the  other  hand  a  smaller-than-necessary  network  faces  difficulty  in  learning  the  training  data.  The  common 
method  of  trial  and  error  is  used  to  design  the  network. 

In  each  case  of  network  design,  a  large  number  of  networks  differing  in  number  of  hidden  nodes  and  memory 
depth  are  trained  and  the  smallest  one  that  learns  the  training  data  with  best  generalization  performance  is  selected.  Table  1 
shows  the  architectural  description  of  the  focused  PRNN  models  with  TDNN,  Gamma  Memory  and  Laguarre  memory 
used  in  the  study  and  without  memory  to  the  input  layer. 
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Table  1:  Architectural  Description  of  PRNN  with  the  Selected  Memories  Fed  into 
Input  Layer  and  without  Any  Memory  to  the  Input  Layer 


Model 

Memory 
Order (K) 

Model 
Architecture 

PRNNG 

K=  1 

08-4-4 

PRNNTD 

K  =  2 

12-3-4 

PRNNL 

K=  1 

08-3-4 

PRNN 

Without 
memory 

04-3-4 

p  =  No  of  input  nodes;  m  =  No  of  hidden  nodes;  n  =  No  of  output  nodes 
Back  propagation  through  time  (BPTT)  algorithm  was  used  to  train  the  PRNN  networks. 
RESULTS 

To  evaluate  the  model  performances  in  forecasting  flows  at  multiple  sections  performance  indices  like  root  mean 
square  error  (RMSE),  coefficient  of  efficiency,  (CE)  and  correlation  coefficient,  (r)  are  used.  The  model  performances 
indices  for  a  forecasting  step  of  2  hours  are  summarized  in  Table  2.  Figure  5  shows  observed  and  predicted  flow  for 
Tarboro  obtained  using  the  PRNN  models.  The  results  given  in  the  Figure  5  and  in  Table  2  indicate  that  focused  PRNN 
without  memory  to  the  input  layer  performs  better  almost  in  all  counts  considered  in  the  study. 

CONCLUSIONS 

Flood  flow  prediction  at  three  upstream  and  a  down  stream  section  of  a  river  network  using  partial  recurrent 
neural  networks  with  and  without  memory  attached  to  the  input  layer  is  accomplished.  Flow  prediction  at  different  stations 
are  obtained  using  partially  recurrent  neural  networks  attached  with  three  different  memories  namely,  TDNN  memory, 
Gamma  memory,  and  Laguarre  memory  at  to  the  input  layer.  The  models  are  applied  to  forecast  flood  flows  in  Tar  basin, 
USA.  Results  obtained  indicates  that  though  there  may  be  difficulties  in  training  a  partially  recurrent  network  having  no 
memory  attached  to  the  network,  the  model  performs  better  in  forecasting  multiple  flows  over  a  basin. 

Table  2:  Comparative  Performance  of  ANN  Models  in  Terms  of  RMSE  (m3/s),  CE  and  r 


Gauging 

Model 

RMSE 

CE 

r 

Gauging 
Station 

Model 

RMSE 

CE 

PRNNG 

62.48 

0.9970 

0.9990 

PRNNG 

42.02 

0.9968 

0.9983 

Rocky 

PRNNTD 

205.71 

0.9760 

0.9885 

Hilliardston 

PRNNTD 

188.34 

0.9422 

0.9753 

Mount 

PRNNL 

72.55 

0.9972 

0.9989 

PRNNL 

47.46 

0.9960 

0.9980 

PRNN 

60.33 

0.9975 

0.9991 

PRNN 

41.63 

0.9969 

0.9986 

PRNNG 

15.16 

0.9893 

0.9936 

PRNNG 

94.94 

0.9950 

0.9993 

Enfield 

PRNNTD 

67.02 

0.9533 

0.9823 

Tarboro 

PRNNTD 

373.89 

0.9783 

0.9893 

PRNNL 

18.72 

0.9964 

0.9982 

PRNNL 

94.01 

0.9986 

0.9993 

PRNN 

15.46 

0.9987 

0.9978 

PRNN 

75.88 

0.9989 

0.9995 
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Figure  5:  Flow  Forecasts  Obtained  for  Tarboro  Using  Different  PRNN  Modes 
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